This invention relates to automatic cueing of speech.
Speechreading (also sometimes referred to as lipreading) is an important means of communication for the deaf and hearing-impaired. By observing the facial actions of the speaker, the listener can derive significant information about the spoken utterance. However, speechreading alone typically provides a limited basis for communication because many distinctions among speech elements are not observable visually. Cueing of speech provides some of the missing information, allowing the receiver to resolve the ambiguities inherent in speechreading.
Manually cued speech is a phonemically-based system of eight consonant and four vowel classes, each class containing between 2 and 4 phonemes. For each syllable the speaker articulates, he indicates the consonant class by a hand shape and the vowel class by hand position near his mouth. Phonemes that appear similar on speakers' lips are placed in different classes (e.g., /p/, /b/, and /m/ belong to separate cue groups). Combinations of the cues synchronized with natural speech movements make spoken language clearly visible and understandable to the speechreader. The cue receiver can identify all spoken phonemes through the combination of visible facial actions and manual cues and understand every spoken syllable that a hearing person hears. This capability can contribute to the reception of spoken language, face to face communication and the development of general language reading skills.
Although manually cued speech provides enormous benefit in the education of the deaf, the assistance it provides in day-to-day communication is limited to situations in which the speaker, or a transliterator, produces the cues. To overcome this limitation, others have developed systems that would derive and display cues similar to manual cued speech using electronic analysis of acoustic speech signals.
One implementation of artificially displayed cues is described in U.S. Pat. No. 4,972,486 by Cornett et al. (hereinafter, "Cornett"). (See also Cornett, O., Beadles, R., & Wilson, B. (1977), "Automatic cued speech", in J. M. Pickett (ed.), Papers from the Research Conference on Speech Processing Aids for the Deaf, pp.224-239.) In the Cornett system, a computer analyzes speech, identifies speech elements in that speech, and determines appropriate cues corresponding to those speech elements. The Cornett system displays virtual image cues corresponding to the identified cues by a pair of seven-segment LED elements, projected in the viewing field of an eyeglass lens. By activating the segments selectively, nine distinct symbol shapes for cueing phonemes are created. The cue groups selected differ from those used in manual cued speech, but, as in manual cued speech, sounds which are difficult to distinguish through speechreading alone occur in different cue groups.